Sequence-to-Sequence Video Captioning with Residual Connected Gated Recurrent Units

نویسندگان

چکیده

Recurrent neural networks have recently emerged as a useful tool in computer vision and language modeling tasks such image video captioning. The main limitation of these is preserving the gradient flow network gets deeper. We propose captioning approach that utilizes residual connections to overcome this maintain by carrying information through layers from bottom top with additive features. experimental evaluations on MSVD dataset indicate proposed achieves accurate caption generation compared state-of-the-art results. In addition, integrated our custom-designed Android application, WeCapV2, capable generating captions without an internet connection.

برای دانلود باید عضویت طلایی داشته باشید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Sequence to Sequence Model for Video Captioning

Automatically generating video captions with natural language remains a challenge for both the field of nature language processing and computer vision. Recurrent Neural Networks (RNNs), which models sequence dynamics, has proved to be effective in visual interpretation. Based on a recent sequence to sequence model for video captioning, which is designed to learn the temporal structure of the se...

متن کامل

Consensus-based Sequence Training for Video Captioning

Captioning models are typically trained using the crossentropy loss. However, their performance is evaluated on other metrics designed to better correlate with human assessments. Recently, it has been shown that reinforcement learning (RL) can directly optimize these metrics in tasks such as captioning. However, this is computationally costly and requires specifying a baseline reward at each st...

متن کامل

Sequence Modeling using Gated Recurrent Neural Networks

In this paper, we have used Recurrent Neural Networks to capture and model human motion data and generate motions by prediction of the next immediate data point at each time-step. Our RNN is armed with recently proposed Gated Recurrent Units which has shown promissing results in some sequence modeling problems such as Machine Translation and Speech Synthesis. We demonstrate that this model is a...

متن کامل

RRA: Recurrent Residual Attention for Sequence Learning

In this paper, we propose a recurrent neural network (RNN) with residual attention (RRA) to learn long-range dependencies from sequential data. We propose to add residual connections across timesteps to RNN, which explicitly enhances the interaction between current state and hidden states that are several timesteps apart. This also allows training errors to be directly back-propagated through r...

متن کامل

Recurrent Residual Learning for Sequence Classification

In this paper, we explore the possibility of leveraging Residual Networks (ResNet), a powerful structure in constructing extremely deep neural network for image understanding, to improve recurrent neural networks (RNN) for modeling sequential data. We show that for sequence classification tasks, incorporating residual connections into recurrent structures yields similar accuracy to Long Short T...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

ژورنال

عنوان ژورنال: Europan journal of science and technology

سال: 2022

ISSN: ['2148-2683']

DOI: https://doi.org/10.31590/ejosat.1071835